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Key findings 


This study examined two main topics: the extent to which teachers in lowa high schools have 

integrated technology with instruction in ways that support the development of 21st century skills 

among students, and how teachers perceive school support for technology use in instruction. 

On technology use: 

~ 44-51 percent of teachers asked students to use technology for collaboration and critical 
thinking, while 22-27 percent asked students to use technology for communication and 
creativity. 
Math teachers were.among the least likely to ask students to use technology for collaboration, 
communication, and creativity. 
78 percent of teachers agreed that technology enhances student learning and that they have 
the ability to integrate technology with instruction. 

On school support for technology use: 

~ 93 percent of teachers reported having access to computers for student use, and 64 percent 
reported that technical support is above average or excellent. 

~ 36 percent of teachers rated the quality of technology-focused professional development as 
above average or excellent. 
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Summary 


The rural districts served by the Central Rivers Area Education Agency (Central Rivers 
AEA)! in Iowa have invested in technology to assist teachers in supporting the devel- 
opment in students of the 21st century skills emphasized by the Iowa Core Standards. 
Central Rivers AEA and three high schools in its service area have raised several questions 
about technology use. They asked Regional Educational Laboratory Midwest to conduct a 
descriptive research study on how much teachers have integrated technology with instruc- 
tion in ways that are believed to support the development of 21st century skills among 
students and how teachers perceive school support for technology use in instruction. 


This study used data from a survey administered in 2017 in 26 public high schools in the 
Central Rivers AEA service area. 


Three research questions concerned teacher use and perceptions of technology. They 
examined how frequently teachers ask students to use technology in ways that are believed 
to support the development of the 21st century skills of collaboration, communication, 
creativity, and critical thinking; how much teachers perceive that technology enhances 
student learning; and how much teachers believe that they have the ability to integrate 
technology with instruction. 


Three research questions concerned school support for technology use. They examined 
how much schools provide technology infrastructure and support, how much teachers par- 
ticipate in high-quality professional development on using technology to support instruc 
tion, and how much school leaders encourage classroom technology use. 


The findings indicate that the proportion of teachers asking students to use technology in 
ways that are believed to support the development of 21st century skills differs across skills, 
subject taught, and years of teaching experience. Key findings on classroom technology use 
include: 

e 89 percent of teachers reported that their students use technology almost daily or 
weekly. 

e 44-51 percent of teachers asked students at least monthly to engage in at least 
two different uses of technology for collaboration and critical thinking, while 
22-27 percent asked students at least monthly to engage in at least two different 
uses of technology for communication and creativity. 

e Math teachers were among the least likely to ask students at least monthly to 
engage in at least two different uses of technology for collaboration, communica- 
tion, and creativity. 

e Teachers with 4-9 years of teaching experience were the most likely to ask stu- 
dents at least monthly to engage in at least two different uses of technology for 
collaboration and creativity, and teachers with 20 or more years of teaching expe- 
rience were the least likely. 


Most teachers believe that technology has specific benefits for student learning and that 
they can integrate technology with instruction. Key findings on teacher perceptions of 
technology include: 
e 78 percent of teachers agreed or strongly agreed that technology enhances student 
learning. Science teachers, career and technical education teachers, and world 


language teachers were the most likely to agree, and arts teachers and math teach- 
ers were the least likely. 

e 78 percent of teachers agreed that they have the ability to integrate technology 
with instruction. Science teachers and teachers with 4—9 years of teaching experi- 
ence were the most likely to agree. 


Most teachers have access to computers, the Internet, and adequate technical support but 
limited access to technology-focused professional development. Key findings on school 
support for technology use include: 
e 93 percent of teachers reported that they could access computers for their students’ 
use all the time. 
© 64 percent of teachers rated technical support as above average or excellent. 
e The typical range for the total number of hours of technology-focused professional 
development that teachers participated in over the last year was 1-8 hours. 
e 36 percent of teachers rated the quality of technology-focused professional devel- 
opment as above average or excellent. 
e About 50 percent of teachers indicated that they discuss technology in profession- 
al settings more than half the time or all the time. 


Although teachers use classroom technology frequently, they are not consistent in using 
it in ways that are believed to support the development of 21st century skills among stu- 
dents and may need additional support and encouragement. Math teachers, in particular, 
may need additional professional development in how to use technology to address spe- 
cific learning objectives. Teachers with 3 or fewer years of teaching experience and those 
with 20 or more years also appear to need additional training on using technology for 
instruction. School leaders planning technology-focused professional learning may wish 
to consider sustained approaches embedded in teachers’ ongoing professional responsibil- 
ities rather than intermittent approaches. To this end, school leaders may consider using 
such formal settings as departmental meetings, evaluations, and classroom observations for 
reflection and planning to reinforce their vision for technology use. 
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Why this study? 


There is growing national consensus on the need for education systems to prepare students 
to succeed in the work environments and society of the 21st century (Partnership for 21st 
Century Skills, 2009; see Voogt & Roblin, 2012, for a review). Recognizing this need, a 
team of Iowa educators and business representatives identified a set of skills essential for 
21st century success (lowa Department of Education, n.d.). Based on that work, the Iowa 
Core Standards were recently revised to emphasize the following skills related to learning 
and innovation: 
e Collaboration: working within and across personal and global networks to achieve 
common goals. 
¢ Communication: sharing information through multiple means, including visual, 
digital, verbal, and nonverbal interactions, leading to an accurate exchange of 
information and ideas. 
e Creativity: generating new or original thoughts, interpretations, products, works, 
or techniques. 
e Critical thinking: accessing and analyzing key information to develop solutions to 
complex problems that may have no clear answer. 


Technology can connect students with learning opportunities for developing and practic- 
ing these skills (Voogt, Erstad, Dede, & Mishra, 2013). Specifically, it can engage students 
in activities that develop collaboration, communication, creativity, and critical thinking. 
Examples of such student activities include: 

e Collaborating online with students at other schools. 

e Writing online reviews, blog posts, and constructive comments. 

e Creating art, music, movies, or webcasts using technology. 

e Collecting and analyzing data from online sources. 


Iowa school districts have invested in technology to address the Iowa Core Standards 
expectations for 21st century skills. For example, the rural districts served by the Central 
Rivers Area Education Agency (Central Rivers AEA)’ have provided a tablet or laptop for 
every student and teacher. 


To promote effective technology use, Central Rivers AEA and three high schools in its 
service area formed the lowa Learning and Technology Networked Improvement Commu- 
nity (Iowa NIC) in partnership with Regional Educational Laboratory (REL) Midwest. In 
spring 2017 the Iowa NIC began testing strategies to help high school educators understand 
how technology can enhance teaching and learning. Along with these efforts, lowa NIC 
participants raised several questions about technology use in their schools. They asked REL 
Midwest to conduct a descriptive research study on how much teachers in schools such as 
theirs integrate technology with instruction in ways that are believed to support the devel- 
opment of 21st century skills among students and on how teachers perceive school support in 
three areas that promote technology use in instruction (these areas are described in box 1). 


The study findings were expected to show the Iowa NIC participants whether teachers 
were using technology in the ways envisioned by the Iowa Core Standards, how much 
school support is available to teachers, how much teachers believe that they have the 
ability to integrate technology with instruction, and how much they believe that doing so 
enhances learning. 


Box 1. Previous research on school support for technology use and teacher 
perceptions of school support for technology use 


Three critical school supports for instructional technology use are technology infrastructure, 
opportunities for professional learning, and encouragement by school leaders (Valiente, 2010). 

A robust technology infrastructure ensures that computer use is reliable and that technical 
problems can be quickly addressed. Technology infrastructure includes broadband Internet 
access, wireless network connectivity, computing device access, and technical support for 
teachers. School districts must also ensure appropriate website filtering policies so that stu- 
dents can reliably access the websites they need for class assignments. Teachers who know 
that their students can reliably access devices and the Internet are more likely to plan lessons 
that incorporate digital resources (Argueta, Huff, Tingen, & Corn, 2011; Center for Promise, 
2013; Inan, Lowther, Ross, & Strahl, 2010; Valiente, 2010). 

High-quality professional learning opportunities help teachers integrate technology with 
instruction in ways that support the development of 21st century skills among students 
(Argueta et al., 2011). Sporadic training sessions on how to use devices and software are 
unlikely to support technology integration in ways that enhance the effectiveness of instruction 
(Center for Public Education, 2013; Kopcha, 2012). But professional development sustained 
throughout the school year and embedded in the school day (for example, through mentoring) 
is associated with changes in teachers’ instructional technology practices (Duran, Brunvand, 
Ellsworth, & Sendag, 2012; Scott & Mouza, 2007). 

Encouragement by school leadership can prompt teachers to use technology in student- 
centered practices that support the development of 21st century skills (Anderson & Dexter, 
2005; Inan et al., 2010; Zhao & Frank, 2003). School leaders need to communicate how they 
expect teachers to use technology. Opportunities for professional discussion, reflection, and 
feedback in existing school routines, such as departmental meetings, evaluations, and class- 
room observations are also critical (see Davies & West, 2014, and Lawless & Pellegrino, 2007, 
for reviews). 

In addition to school supports, teachers’ own perceptions of technology are associated 
with technology use. To the extent that teachers perceive technology in instruction as directly 
relevant to their curriculum goals (Ertmer, Ottenbreit-Leftwich, Sadik, Sendurur, & Sendurur, 
2012) and favorable to student learning (Hur, Shannon, & Wolf, 2016), they are likely to inte- 
grate it. And teachers’ confidence in their ability to integrate technology with instruction is asso- 
ciated with their likelihood of doing so (Anderson, Groulx, & Maninger, 2011; Hur et al., 2016). 

Both school and teacher factors influence teacher perceptions of technology integration 
with instruction. School factors include enrollment size and student composition. Teachers in 
smaller schools (Hsu, Wu, & Hwang, 2007) and teachers in schools with a higher proportion 
of economically disadvantaged students (Perrotta, 2013) have more positive beliefs about the 
potential benefits of technology use. Teacher factors include subject taught and years of teach- 
ing experience. Different subjects vary in how they are supported by technology, so subject 
taught is likely to influence perceptions (Howard, Chan, Mozejko, & Caputi, 2015; Perrotta, 
2013). Years of teaching experience is negatively associated with teachers’ confidence in their 
ability to integrate technology (Inan & Lowther, 2010; Liu, Ritzhaupt, Dawson, & Barron, 2017). 


What the study examined 


The study used data from a survey about technology use for instructional purposes, school 
support for technology use, and teacher beliefs and attitudes that was administered in 2017 


by the Central Rivers AEA to 26 public high schools in its service area. Six research ques- 
tions in two broad categories guided the study: 


On technology use 
e How frequently do teachers ask students to use technology in ways that are 
believed to support the development of the 21st century skills of collaboration, 
communication, creativity, and critical thinking among students? 
¢ How much do teachers perceive that technology enhances student learning? 
e How much do teachers believe that they have the ability to integrate technology 
with instruction? 


On school support for technology use 
¢ How much do schools provide technology infrastructure and support? 
¢ How much do teachers participate in high-quality professional development on 
using technology to support instruction? 
e How much do school leaders encourage classroom technology use? 


The study reports findings across all responding teachers and across groups of teachers 
classified by: 

e Subject taught. 

e Years of teaching experience. 

e School size. 
School Title I status.’ 


The study team conducted a statistical analysis to identify group differences. See box 2 for 
a summary of the data and methods and appendix A for details. 


Box 2. Data and methods 


Instrument and data 

The study used data collected in 2017 through the Clarity Technology and Learning Survey of 
teachers, an instrument included within an online school management system. Central Rivers 
AEA provides all schools with access to this system for monitoring and supporting instructional 
technology practices. The survey includes 132 items related to four domains: access to tech- 
nology resources, teacher technology skills, classroom technology practices, and the school 
technology environment. Forty-five of these items related to the study’s research questions 
and topics (see appendix A for a description of topics and items). 

The survey covered 11 topics, each of which was associated with one or more survey items 
(see table A1 in appendix A). For each research question, the survey team examined respons- 
es for one or more topics. For example, for the research question about school leaders encour- 
aging classroom technology use, the study team examined responses for two topics: frequency 
of discussions of technology use and overall perception of whether the school encourages 
technology use. The topic of frequency was associated with three survey questions, and the 
topic of overall perception was associated with one survey question. 

Subject taught and years of teaching experience were determined based on teachers’ survey 
responses. School size and Title | status were determined based on publicly available data. 


(continued) 


Box 2. Data and methods (continued) 


Survey sample 
All high schools in the service area of the Central Rivers Area Education Agency (Central Rivers 
AEA) have access to the Clarity Technology and Learning Survey. In 2017 the Central Rivers 
AEA communicated with all high schools to remind them of the survey’s availability. Each 
school decided whether to administer the survey to its teachers and chose how to encourage 
teacher participation. The Central Rivers AEA shared deidentified data from the survey for 
2017 with the study team. 

Of the 57 high schools in the Central Rivers AEA service area, 26 administered the survey 
in 2017, for a school response rate of 46 percent. Of 792 possible respondents at those 26 
schools, 524 teachers responded, for a response rate of 66 percent. Survey respondents 
represented a range of subjects taught and years of teaching experience, and they belonged to 
schools that differed in size, Title | status, and locale (See table B1 in appendix B). 


Analysis of nonresponse bias 

Schools that administered the survey were compared with schools that did not on school-level 
academic proficiency, demographics, graduation rates, attendance rates, and college enroll- 
ment rates. None of these variables was related to whether a school administered the survey 
(see table A3 in appendix A). Furthermore, among responding schools, the teacher response 
rate at a school was not significantly correlated with survey topics related to technology use or 
perceptions of technology use (see table A4 in appendix A). 

Teacher response rates differed significantly by subject taught, with teachers in core sub- 
jects (math, English language arts, science, and social studies) most likely to respond to the 
survey (see figure A1 in appendix A). To account for the possible bias introduced by differences 
in participation rates by subject taught, the subsequent survey analysis examined whether 
responses differed by subject taught. 


Data analysis 

The study team performed preliminary analyses to determine whether the survey items in each 
topic could be combined to form a reliable and valid measure, meaning that the individual 
survey items are closely related and measure a single topic. The study team found that items 
for the four 21st century skills (collaboration, communication, creativity, and critical thinking) 
could not be combined to form reliable measures but that items for the following topics could: 
teacher beliefs about how much technology enhances student learning, teacher beliefs about 
how much they have the ability to integrate technology with instruction, quality of technical 
support, duration of technology-focused professional development, quality of technology- 
focused professional development, and frequency of discussions of technology use. Therefore, 
the study team treated the survey data with two different approaches: reporting raw frequen- 
cies and proportions for the four 21st century skills items and for the topics for which there 
was a single survey item and reporting numerical scale scores for the other topics. 

The study team examined the frequency of teacher responses to the items addressing tech- 
nology use for the development of the four 21st century skills. The study team also calculated 
the proportion of teachers emphasizing each skill, defined as asking students at least monthly to 
engage in two or more uses of technology for a given skill, for all teachers and for groups of teach- 
ers classified by subject taught, years of teaching experience, school size, and school Title | status. 
The study team performed additional tests to determine which groups differed in their responses. 


(continued) 


Box 2. Data and methods (continued) 


For the topics for which a reliable scale score could be calculated, the study team cat- 
egorized respondents according to the response option they were most likely to select for 
the topic. These categorizations were used to describe the pattern of findings for each topic. 
The study team tested whether the mean scale scores of groups of teachers differed sig- 
nificantly (using a one-way analysis of variance) and performed additional tests to determine 
which groups of teachers differed in their responses for a given topic. The study team did not 
examine differences by school locale because a large proportion of schools were classified as 
rural (73 percent) or town (19 percent) (See table B2 in appendix B). 


What the study found 


The proportion of teachers asking students to use technology in ways that are believed to 
support the development of 21st century skills differs by skill and across subjects taught 
and years of teaching experience. Most teachers believe that technology enhances student 
learning and that they have the ability to integrate technology with instruction. Most 
teachers have access to computers, the Internet, and adequate technical support but limited 
access to technology-focused professional development. About half of teachers reported 
that discussions of classroom technology use occur frequently in professional settings. 


Most teachers report that students use technology in the classroom almost daily or weekly 
About 89 percent of teachers reported that students use classroom technology, such as 


desktop computers, laptops, or tablets, almost daily or weekly (figure 1; see also table C1 in 
appendix C). 


Figure 1. Nearly 9 in 10 teachers reported that students use classroom technology 
almost daily or weekly 


Percent 
100 7 


75 7 


50 4 


25 4 


oO — 
Almost daily Weekly Monthly Every few months Never 


Note: Data presented in the figure are raw survey results. No statistical testing was performed on these data. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 


Between 22 percent and 51 percent of teachers asked students at least monthly to use technology 
to promote skills of collaboration, communication, creativity, and critical thinking 


About 51 percent of teachers asked students at least monthly to engage in at least two dif- 
ferent uses of technology for collaboration, and 44 percent of teachers asked students to do 
so for critical thinking (figure 2). By contrast, about 27 percent of teachers asked students 
at least monthly to engage in at least two different uses of technology for communication, 
and 22 percent of teachers did so for creativity. 


About 62 percent of teachers asked students at least monthly to collaborate using online 
documents, and 45 percent of teachers asked students to collaborate online with class- 
mates (with the medium of collaboration unspecified; figure 3). By contrast, 37 percent of 
teachers reported asking students at least monthly to collaborate online with the teacher, 
and 7 percent of teachers asked students to collaborate with students in other schools. 


Fewer than 20 percent of teachers asked students at least monthly to use technology to 
engage in direct communication—including receiving feedback from classmates or an 
outside source, writing for an online audience, and chatting online (figure 4). With respect 
to indirect means of communication, 35 percent of teachers asked students to post school- 
work online, and 25 percent asked students to use web tools to receive information. 


About 37 percent of teachers asked students at least monthly to develop multimedia pre- 
sentations. By contrast, fewer than 20 percent of teachers asked students to use technology 
to create art, music, movies, or webcasts (18 percent); use a digital camera (15 percent); or 
create online models, simulations, and animations (13 percent) (figure 5). 


Figure 2. Between 22 percent and 51 percent of teachers asked students at 
least monthly to engage in at least two different uses of technology related to 
collaboration, communication, creativity, and critical thinking 


Percent 
100 - 


75 4 


50 4 


25 4 


oO nt 
Collaboration Critical thinking Communication Creativity 


Note: Data presented in the figure are raw survey results and refer to the percentage of teachers who reported 
asking students weekly or monthly to engage in at least two different uses of technology. No statistical testing 
was performed on these data. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 
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Figure 3. Between 7 percent and 62 percent of teachers asked students at least 
monthly to collaborate online in various ways 


Percent 
100 - 


Collaborate using Collaborate online Collaborate online Collaborate online 
online documents with classmates with teacher with other schools 


Note: Data presented in the figure are raw survey results and refer to the percentage of teachers who reported 
asking students weekly or monthly to engage in each use of technology for collaboration. No statistical testing 
was performed on these data. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 


Figure 4. Between 5 percent and 35 percent of teachers asked students at least 
monthly to use technology to communicate in various ways 


Percent 
100 - 
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50 7 
- : 
; ie 2 a eral = 
Post schoolwork Use web tools Receive Write for Receive feedback Chat 
online to receive feedback from online from outside online 
(ePortfolios) information classmates audience source 


Note: Data presented in the figure are raw survey results and refer to the percentage of teachers who reported 
asking students weekly or monthly to engage in each use of technology for communication. No statistical 
testing was performed on these data. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 


Figure 5. Between 13 percent and 37 percent of teachers asked students at least 
monthly to use technology to create various types of products 


Percent 
100 - 
7574 
50 
25 7 
0 — 
Develop Create art, Usea Create models, 
multimedia music, movies, digital camera simulations, 
presentations webcasts animations 


Note: Data presented in the figure are raw survey results and refer to the percentage of teachers who reported 
asking students weekly or monthly to engage in each use of technology for creativity. No statistical testing was 
performed on these data. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 


About 62 percent of teachers asked students at least monthly to use technology for critical 
thinking by conducting research online, and about 35 percent of teachers asked students 
to do so by collecting and analyzing data or by identifying and solving authentic problems. 
About 24 percent of teachers asked students at least monthly to use technology for critical 
thinking by taking measurements or conducting experiments (figure 6). 


Math teachers and teachers with the most years of teaching experience were the least likely to ask 
students to use technology in ways that are believed to support the development of 21st century skills 


For all four 21st century skills the frequency of teachers asking students to use technology 
varied by subject taught (see table D1 in appendix D). And for collaboration and creativity 
the frequency varied by years of teaching experience. The frequency did not vary based on 
school size or Title I status. 


Math, arts, and physical education teachers were the least likely to ask students at 
least monthly to engage in at least two different uses of technology for collaboration. 
Between 55 percent and 75 percent of career and technical education, science, world lan- 
guage, English language arts, and social studies teachers asked students at least monthly 
to use technology to collaborate in at least two different ways (figure 7; see also table C2 
in appendix C). By contrast, 32 percent of physical education teachers, 17 percent of arts 
teachers, and 16 percent of math teachers did. 


Math teachers were the least likely to ask students at least monthly to engage in at least 
two different uses of technology for communication. Between 22 percent and 46 percent 
of arts, career and technical education, science, world language, English language arts, and 
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Figure 6. Between 24 percent and 62 percent of teachers asked students at least 
monthly to use technology for critical thinking in various ways 


Percent 
100 - 
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50 4 
25 7 
0) — 
Conduct research Collect and Identify and Take measurements 
online analyze data solve authentic or conduct 
problems experiments 


Note: Data presented in the figure are raw survey results and refer to the percentage of teachers who reported 
asking students weekly or monthly to engage in each use of technology for critical thinking. No statistical 
testing was performed on these data. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 


Figure 7. Math, arts, and physical education teachers were the least likely to ask 
students at least monthly to use technology for collaboration 
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Note: Data are raw survey results and refer to the percentage of teachers who reported asking students 
weekly or monthly to engage in at least two different uses of technology for collaboration. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 


social studies teachers asked students at least monthly to engage in at least two different 
uses of technology for communication (figure 8; see table C3 in appendix C). By contrast, 
16 percent of physical education teachers and 3 percent of math teachers did. 


Math teachers were the least likely to ask students at least monthly to engage in at 
least two different uses of technology for creativity. Between 22 percent and 42 percent 
of arts, world language, social studies, career and technical education, and science teachers 
asked students at least monthly to engage in at least two different uses of technology for 
creativity (figure 9; see also table C4 in appendix C). By contrast, 12 percent of physical 
education and English language arts teachers and 3 percent of math teachers did. 


Science teachers and career and technical education teachers were the most likely to 
ask students at least monthly to engage in at least two different uses of technology 
for critical thinking. About 72 percent of science teachers and 58 percent of career and 
technical education teachers asked students at least monthly to use technology for critical 
thinking (figure 10; see table C5 in appendix C). By contrast, fewer than 20 percent of 
physical education, world language, and arts teachers did. 


Teachers with 4—9 years of teaching experience were more likely than teachers with 20 
or more years of teaching experience to ask students at least monthly to engage in at 
least two different uses of technology for collaboration or for creativity. About 62 percent 
of teachers with 4-9 years of teaching experience asked students at least monthly to engage 
in at least two different uses of technology for collaboration, compared with 40 percent of 
teachers with 20 or more years of teaching experience (figure 11; see table C2 in appen- 
dix C). In addition, 57 percent of teachers with 3 or fewer years of teaching experience 


Figure 8. Math teachers were the least likely to ask students at least monthly to 
use technology for communication 
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Note: Data are raw survey results and refer to the percentage of teachers who reported asking students 
weekly or monthly to engage in at least two different uses of technology for communication. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 
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Figure 9. Math teachers were the least likely to ask students at least monthly to 
use technology for creativity 


Percent 
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Note: Data are raw survey results and refer to the percentage of teachers who reported asking students to 
engage weekly or monthly in at least two different uses of technology for creativity. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 


Figure 10. Science teachers and career and technical education teachers were the 
most likely to ask students at least monthly to use technology for critical thinking 
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Note: Data are raw survey results and refer to the percentage of teachers who reported asking students 
weekly or monthly to engage in at least two different uses of technology for critical thinking. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 


Figure 11. Teachers with 4-9 years of teaching experience were more likely than 
teachers with 20 or more years of teaching experience to ask students at least 
monthly to use technology for collaboration 
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Note: Data are raw survey results and refer to the percentage of teachers who reported asking students 
weekly or monthly to engage in at least two different uses of technology for collaboration. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 


asked students at least monthly to engage in at least two different uses of technology for 
collaboration, as did 54 percent of teachers with 10-19 years of teaching experience. 


About 35 percent of teachers with 4-9 years of teaching experience reported asking stu- 
dents at least monthly to engage in at least two different uses of technology for creativity, 
compared with 14 percent of teachers with 20 or more years of experience (figure 12; see 
table C4 in appendix C). In addition, 25 percent of teachers with 3 or fewer years of teach- 
ing experience asked students at least monthly to engage in at least two different uses of 
technology for collaboration, as did 22 percent of teachers with 10-19 years of teaching 
experience. 


Nearly four in five teachers agreed that technology enhances student learning and that they have 
the ability to integrate technology with instruction 


About 78 percent of teachers agreed or strongly agreed that technology enhances student 
learning, 21 percent expressed a neutral opinion, and 1 percent disagreed or strongly dis- 
agreed (figure 13; see also table C6 in appendix C). In addition, 78 percent of teachers 
agreed or strongly agreed that they have the ability to integrate technology with instruc- 
tion, 15 percent expressed a neutral opinion, and 7 percent disagreed or strongly disagreed 
(see figure 13; see also table C7 in appendix C). 


Teachers’ agreement that technology enhances learning differed by subject taught and 
school size (but not by years of teaching experience or school Title I status), and teachers’ 
agreement that they have the ability to integrate technology with instruction differed by 
subject taught and years of teaching experience (but not by school size or Title I status) 
(see table D2 in appendix D). 


Figure 12. Teachers with 4-9 years of teaching experience were more likely than 
teachers with 20 or more years of teaching experience to ask students at least 
monthly to use of technology for creativity 


Percent 
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0 = 
3 or fewer years 4-9 years 10-19 years 20 or more years 


Note: Data are raw survey results and refer to the percentage of teachers who reported asking students 
weekly or monthly to engage in at least two different uses of technology for creativity. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 


Figure 13. Nearly four in five teachers agreed that technology enhances student 
learning and that they have the ability to integrate technology with instruction 


Percent 
100 - 


Perceived ability to Perceived enhancement 
integrate technology of student learning 


Note: Data presented in the figure are scale score summaries (See box 2) and refer to the percentage of 
teachers who answered agree or strongly agree on the scale. No statistical testing was performed on these 
data. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 


Science teachers were the most likely to agree that technology enhances student learn- 
ing and that they have the ability to integrate technology with instruction. About 
86 percent of science teachers and 85 percent of career and technical education teachers 
and world language teachers agreed or strongly agreed that technology enhances student 
learning (figure 14; see also table C6 in appendix C). By contrast, 57 percent of arts teach- 
ers and 67 percent of math teachers agreed or strongly agreed. 


About 89 percent of science teachers agreed or strongly agreed that they have the ability 
to integrate technology with instruction, compared with 68 percent of math teachers 
(figure 15; see also table C7 in appendix C). 


Teachers with 20 or more years of teaching experience were the least likely to agree 
that they have the ability to integrate technology with instruction. About 68 percent of 
teachers with 20 or more years of teaching experience agreed or strongly agreed that they 
had the ability to integrate technology with instruction (see table C7 in appendix C). By 
contrast, about 87 percent of teachers with 4—9 years of experience did. 


Teachers at small schools were more likely to agree that technology enhances student 
learning than were teachers at large schools. About 83 percent of teachers at small 
schools (those with fewer than 300 students) agreed or strongly agreed that technology 
enhances student learning, compared with 70 percent of teachers at large schools (those 
with more than 500 students) (see table C6 in appendix C). 


Figure 14. Science teachers were the most likely to agree that technology 
enhances student learning 
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Note: Data are scale score summaries (See box 2) and refer to the percentage of teachers who answered 
agree or strongly agree on the scale. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 
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Figure 15. Science teachers were the most likely to agree that they have the 
ability to integrate technology with instruction 


Percent 
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education studies language language technical 
arts education 


Note: Data are scale score summaries (See box 2) and refer to the percentage of teachers who answered 
agree or strongly agree on the scale. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 


Most teachers reported having access to computers and websites needed for student use 


About 93 percent of teachers reported having access to computers for student use all 
the time (see table C8 in appendix C). School Internet filters do not appear to pose a 
barrier to technology use. Most teachers reported that their school’s Internet filters rarely 
(49 percent) or never (14 percent) prevented access to websites needed for class. Most of 
the remaining teachers (27 percent) indicated that filters limited access less than half the 
time (see table C9). 


A majority of teachers rated technical support as above average or excellent, but fewer than half of 
teachers rated technology-focused professional development as above average or excellent 


About 64 percent of teachers rated the quality of technical support they received from 
their school on issues pertaining to Internet connectivity, hardware, and troubleshooting 
of disruptions to instruction as above average or excellent, and 35 percent rated the quality 
as average (figure 16). Teachers at small schools rated technical support higher than did 
teachers at large schools (see table C10 in appendix C and table D2 in appendix D). No 
differences were evident across subject taught, years of teaching experience, or school Title 
I status. 


Teacher ratings of technology-focused professional development were lower than ratings of 
technical support. About 55 percent of teachers rated the quality of three types of formal 
and informal technology-focused professional development that they completed during the 
12 months prior to the survey as average, while 36 percent rated them as either above 
average or excellent (see figure 16). Teachers with 4-9 years of teaching experience were 
the least likely to rate technology-focused professional development as above average or 
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Figure 16. About 64 percent of teachers rated as above average or excellent the 
quality of technical support from their school, whereas 36 percent of teachers 
rated as above average or excellent the quality of technology-focused professional 
development 


Percent 
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Note: Data presented in the figure are scale score summaries (See box 2) and refer to the percentage of 
teachers who answered above average or excellent. No statistical testing was performed on these data. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 


excellent (see table C11 in appendix C). No differences were evident across subject taught, 
school size, or school Title I status (see table D2 in appendix D). 


More than half of teachers reported spending 1-8 hours in technology-focused professional 
development during the past year 


About 55 percent of teachers reported spending 1-8 hours in formal and informal 
technology-focused professional development during the 12 months prior to the survey, 
18 percent reported spending 17-32 hours, and 11 percent reported spending 9-16 hours* 
(figure 17; see also table C12 in appendix C). 


Half of teachers reported that discussions of instructional technology occur frequently 


About 50 percent of teachers reported that discussions of technology in formal profession- 
al settings, such as department meetings, evaluations, and classroom observations or visits, 
occurred more than half the time or always. By contrast, 19 percent of teachers reported 
that such discussions rarely or never occur (figure 18; see also table C13 in appendix C). 
Science teachers, career and technical education teachers, and social studies teachers were 
the most likely to report frequent discussions of technology use. The perceived frequency 
of these discussions did not differ across years of teaching experience, school size, or school 
Title I status (see table D2 in appendix D). 
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Figure 17. More than half of teachers reported spending 1-8 hours in technology- 
focused professional development during the past year 


Percent 
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None 1-8 hours 9-16 hours 17-32 hours 33 or more hours 


Note. Data are scale score summaries (See box 2). No statistical testing was performed on these data. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 


Figure 18. Half of teachers reported that discussions of instructional technology 
occur frequently in professional settings 


Percent 
100 - 


Always More than half Less than half Rarely Never 
of the time of the time 


Note: Data are scale score summaries (See box 2). No statistical testing was performed on these data. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 


Implications of the study findings 


These findings suggest that although students frequently use technology in the classroom, 
they are not frequently using technology in ways that are believed to support the devel- 
opment of the 21st century skills of collaboration, communication, creativity, and critical 
thinking. Because the Clarity Technology and Learning Survey does not include a com- 
prehensive set of technology uses, the study cannot describe how students use technology 
in the classroom for other objectives or how they use other technology for the same objec- 
tives. However, past research has indicated that teachers frequently use classroom technol- 
ogy to present instructional content, a use that is unlikely to support the development of 
21st century skills (Inan et al., 2010). Focusing professional learning on building teachers’ 
understanding and abilities to use technology in ways that support the development of 
these skills offers an important opportunity. 


Math teachers reported lower levels of technology integration than did teachers of other 
subjects and may need additional professional development in using technology to address 
specific learning objectives. Teachers’ ability to integrate technology with instruction 
requires specialized knowledge of how technology can support content-specific pedagogies 
or instructional goals (Voogt, Fisser, Pareja Roblin, Tondeur, & van Braak, 2013). Math 
teachers also may need support finding content-specific digital resources (Handal, Camp- 
bell, Cavanagh, Petocz, & Kelly, 2013). 


Teachers with 4-9 years of teaching experience were the most likely to ask students to 
use technology in ways that are believed to support the development of 21st century skills 
and to agree that they have the ability to integrate technology with instruction, while 
teachers with 20 or more years of teaching experience were the least likely. This pattern 
may reflect that recently minted teachers have received more extensive training in the 
use of educational technology (Inan & Lowther, 2010) or that many current technologies 
did not exist when the most experienced teachers completed their preservice training. At 
the same time, teachers with 3 or fewer years of teaching experience were less likely to use 
technology in ways that are believed to support the development of 21st century skills. It 
is possible that their preservice training introduced them to technology applications but 
did not explain how to integrate them with pedagogical strategies (Chai, Koh, & Tsai, 
2010). The findings suggest that both the least and the most experienced teachers need 
additional professional development and support for designing and teaching lessons that 
use technology to enhance the learning of subject matter content (Niess, 2011). 


School and district leaders could provide teachers with ongoing technology-focused pro- 
fessional learning that is connected to their day-to-day classroom work. The finding that 
most teachers receive eight or fewer hours of technology-focused professional learning a 
year suggests that such professional learning is sporadic rather than sustained. A broad 
research consensus suggests that to be effective, technology-focused professional develop- 
ment must be sustained and directly relevant to teachers’ content-specific instructional 
practice. For example, instructional coaches could help teachers consider how technolo- 
gy could support or enhance learning activities that are appropriate for teaching certain 
topics (Croft, Coggshall, Dolan, & Powers, 2010; Davies & West, 2014; Jang, 2010; Kopcha, 
2012; see also Darling-Hammond & Rothman, 2015). 
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Similarly, school leaders have an opportunity to embed professional discussions of technolo- 
gy into such formal settings as departmental meetings, evaluations, and classroom observa- 
tions. With only about half of teachers in the study indicating that they frequently discussed 
technology in these settings, this opportunity has not yet been fully realized. Such discus- 
sions could reinforce the school vision for technology use through reflection and planning. 


The study findings indicate that technical infrastructure and support are adequate for sup- 
porting instructional technology practices. Most teachers reported high access to com- 
puting devices and the Internet. And most rated technical support as above average or 
excellent, with most of the remainder rating it as average. However, district and school 
leaders may want to consider whether technology resources and support are sufficient for 
technology use related to communication and creativity because those uses appeared to be 
less frequent than other uses evaluated in this study. 


Whether ongoing and high-quality professional development is associated with better 
technology integration in instruction remains unknown, though past research suggests the 
importance of such school- and teacher-level factors (Hur et al., 2016; Inan & Lowther, 
2010). Additional analysis of the Clarity Technology and Learning Survey data could help 
schools prioritize areas for improvement by examining what supports appear to be associat- 
ed with higher rates of technology integration. 


Limitations of the study 


The sample of participating schools is not necessarily representative of all schools in the 
region served by the Central Rivers AEA. Individual schools decided whether to adminis- 
ter the survey to monitor technology implementation. Although the responding and non- 
responding schools did not differ on demographic and academic characteristics for which 
data are publicly available, the two groups may differ on other characteristics, such as lead- 
ership style and school culture, that could be related both to a school’s nonresponse and to 
its overall level of implementation of educational technology. 


Similarly, the sample of teachers who responded to the survey may not be representative 
of the population of teachers within the participating schools. The only characteristic 
for which data were available for comparing the sample of teachers to the population of 
teachers was subject taught. Teachers in core subjects (math, English language arts, social 
studies, and science) were the most likely to respond to the survey. Data on other char 
acteristics of nonresponding teachers (for example, gender, age, and attitudes about the 
school) were not available to the study team. 


Differences in technology use by subject taught may be due partly to the design of the 
survey instrument rather than to actual differences in the frequency of teachers asking 
students to use technology in ways that are believed to support the development of 21st 
century skills. Specifically, several technology uses relate primarily to science instruction, 
such as “Take measurements or do experiments using technology.” Since other subjects 
may not equally emphasize requiring students to take measurements or do experiments, 
teachers in those areas could be less likely to ask students to use technology for those 
tasks. So, the higher proportion of science teachers reporting technology use in the study 
may relate to the better match of their subject to the types of technology uses the survey 


addressed. 
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Appendix A. Study methodology 


This appendix summarizes the study’s methodology for collecting survey data, processing 
the data to create variables, examining response rates for bias related to school and teacher 
nonresponse, and analyzing survey data to identify meaningful differences across respon- 
dents from different groups. 


Survey data collection 


Questionnaire design. The Clarity Technology and Learning Survey is administered to 
both teachers and students through an online platform. The full teacher survey includes 
140 items, of which 45 items aligned with the research questions of this study (see appen- 
dix E). Items correspond to a specific topic related to the research questions (table A1). 


To summarize the topics: 

e For frequency of student use of computing devices, the survey included a single 
item. Teachers could select from response options of almost daily, weekly, monthly, 
every few months, or never. 

e For frequency of asking students to use technology in ways that support 21st 
century skills, the survey included 18 items referring to collaboration, communica- 
tion, creativity, and critical thinking. Teachers could select from response options 
of weekly, monthly, every few months, or never. 


Table A1. Topics addressing each research question and the number of survey 
items per topic 


Number 

Research question Survey topic of items 
How frequently do teachers ask students to Frequency of student use of computing devices 4 
use technology in ways that are believed to Frequency of asking students to use technology 
support the development of the 21st century jy ways that are believed to support the 
skills of collaboration, communication, development of 21st century skills 18 
Clear Hy rond Craeal Thikine? Items related to collaboration 4 

Items related to communication 6 

Items related to creativity 4 

Items related to critical thinking 4 
How much do teachers perceive that Teachers’ beliefs about how much technology 
technology enhances student learning? enhances student learning 3 
How much do teachers believe that they Teachers’ beliefs about how much they have the 
have the ability to integrate technology with ability to integrate technology with instruction 
instruction? 4 
How much do schools provide technology Access to computers for student use 1 
infrastructure and support? School filters preventing access to websites 4 

Quality of technical support 7 
How much do teachers participate in high- Duration of technology-focused professional 
quality professional development on using development 3 
technology to support instruction? Quality of technology-focused professional 
development 3 

How much do school leaders encourage Frequency of discussions of technology use 3 
classroom technology use? School encourages technology use 4 


Source: Authors’ compilation. 
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e For teachers’ beliefs about the extent to which technology enhances student learn- 
ing, the survey included three items referring to specific benefits (such as increased 
student engagement). Teachers could select from response options of strongly 
agree, agree, are neutral, disagree, or strongly disagree. 

e For teachers’ beliefs about their ability to integrate technology with instruction, 
the survey included four items about teacher confidence in performing related 
tasks (such as identify relevant digital resources). Teachers could select from 
response options of strongly agree, agree, are neutral, disagree, or strongly disagree. 

e For access to computers for student use, the survey included a single item. Teachers 
could select from response options of all of the time, over half of the time, less 
than half of the time, rarely, or never. 

e For school filters preventing access to websites, the survey included a single item. 
Teachers could select from response options of never, rarely, less than half of the 
time, more than half of the time, or all of the time. 

e For quality of technical support, the survey included seven items asking teachers 
to rate the quality of support for different types of equipment or infrastructure. 
Teachers could select from response options of excellent, above average, average, 
below average, poor, or not applicable. 

e For duration of technology-focused professional development, the survey included 
three items on different types of formal and informal activities. Teachers could 
select from response options of 33 or more hours, 17-32 hours, 9-16 hours, 1-8 
hours, or none. 

e For quality of technology-focused professional development, the survey includ- 
ed the same three items covered for duration of technology-focused professional 
development, but teachers rated their perceived quality of the professional devel- 
opment. Teachers could select from response options of excellent, above average, 
average, below average, poor, or I didn’t do this. 

e For frequency of discussions of technology use, the survey included three items on 
different professional settings (department meetings, evaluations, and classroom 
observations or visits). Teachers could select from response options of all of the 
time, more than half of the time, less than half of the time, rarely, or never. 

e For school encouragement of technology use for teaching and learning, the survey 
included a single item. Teachers could select from response options of strongly 
agree, agree, are neutral, disagree, or strongly disagree. 


Data collection. The Central Rivers Area Education Agency (Central Rivers AEA) 
service area contains 57 public secondary schools. All these schools had access to the 
Clarity Technology and Learning Survey and received communication from the Central 
Rivers AEA alerting them about the resource and its benefits. Each school decided whether 
to administer the survey to its teachers and how to encourage teacher participation. Of 
the 57 schools, 26 administered the survey to teachers in 2017. The Central Rivers AEA 
shared deidentified data from teacher surveys completed in 2017 with the study team in 
February 2018. 


Data processing 
The study team processed and cleaned the survey data in preparation for analysis. Post- 
collection processing included reviewing each survey item for internal consistency, consis- 


tency between related items, computation of scale scores, and nonresponse patterns. The 
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study team also reviewed missing data to identify whether survey nonresponse was associ- 
ated with school and teacher characteristics. 


Internal consistency. The study team used a Rasch measurement model to calculate 
person reliability statistics to determine whether items in a topic reflected the same under 
lying construct (Wright & Masters, 1982). Specifically, the study team calculated seven 
coefficient alphas for the seven topics that comprised three or more items. The benchmark 
for adequate internal reliability was set to .70 for the Rasch reliability coefficient. The reli- 
ability coefficient for most topics met or exceeded the benchmark. However, the two topics 
related to professional development (hours spent in technology-focused professional devel- 
opment and quality of technology-focused professional development) had internal reliabil- 
ities that were below but approaching .70. Those two scales have only three items, and 
scales with fewer items tend to have lower reliability estimates related to the way Rasch 
reliability is calculated. Given this consideration, as well as the fact that the coefficient 
alpha was high for the topic related to quality of technology-focused professional develop- 
ment, the study team calculated scale scores for these two topics. 


The study team took a different approach regarding the frequency of asking students to use 
technology in ways that are believed to support the development of the four 21st century 
skills of collaboration, communication, creativity, and critical thinking. Three of the four 
skills had Rasch reliabilities far less than .70 (for example, .32 for communication), sug- 
gesting that an aggregate score would be unreliable because the Rasch model could not 
distinguish persons throughout most of the distribution. Therefore, the study team did not 
develop scale scores for this topic. Instead, the study team applied an operationalized def- 
nition of teachers “emphasizing a skill” as asking students at least once a month to engage 
in two or more uses of technology for that skill. With this definition, the study team cal- 
culated the proportion of teachers who emphasized a specific skill and the proportion who 
did not, providing an avenue for comparing teachers’ responses regarding the 21st century 
skills topics across different groups. 


Developing the scale scores. With the application of the Rasch model, the study team 
calculated scale scores for all topic areas in the teacher survey besides the four 21st century 
skills. The Rasch scores were created using a one-parameter rating scale model using Win- 
steps. The rating scale model was chosen instead of the partial credit model because all 
items in a scale have the same rating categories, and, therefore, it was reasonable to assume 
that the distances between categories were constant across items. Rasch scores were gener- 
ated separately for each scale and for all teachers. Item threshold parameters were estimat- 
ed simultaneously as the scores. 


For the topics for which it was possible to calculate a reliable scale score, the study team 
categorized respondents according to the response option they were most likely to select 
for the topic. These categorizations were used to describe the pattern of findings for each 
topic. In each scale the study team generated threshold estimates for each item as part of 
the Rasch modeling and using the procedure described by Garet, Heppen, Walters, Smith, 
& Yang (2016). Because all the items have the same number of rating categories and thus 
the same number of threshold parameters, the study team calculated the average threshold 
parameter for each category across all items, generating an “average” item with the average 
threshold parameters for each scale. Using ability estimates (Rasch scores) and the thresh- 
old parameters for the “average” item, the study team generated—for each teacher and 
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for each scale—the probability of scoring at each rating scale level. The study team then 
categorized respondents by their highest probable response category. Rasch scores, which 
are conceptually equivalent to the most probable responses of a scale, were used to perform 
the between-group difference tests. The most probable responses of a scale were presented 
in charts to illustrate between-group differences. 


Response rate and nonresponse bias analysis 


The study achieved a school-level response rate of 46 percent (26 out of 57 possible schools) 
and a teacher-level response rate of 66 percent (524 out of 792 possible respondents). 
Because the school- and teacher-level nonresponse rates were greater than 30 percent, the 
study team conducted nonresponse bias analysis that focused on three questions: 

¢ Doschools that participated in the survey differ from schools that did not? 

e Do low response rates in a school reflect a sample biased toward more frequent 

users of technology? 
e Do response rates differ as a function of subject area taught? 


Do schools that participated in the survey differ from schools that did not? To evalu- 
ate whether school-level characteristics were associated with a school’s nonparticipation 
in the survey, the study team used a logistic regression in which the predictor variables 
were school-level academic proficiency, demographics, graduation rates, attendance rates, 
and college enrollment rates (table A2). Treating whether schools administered the survey 
as a binary outcome, logistic regression was performed to determine whether school-level 
characteristics were associated with the outcome of administering the survey. The logistic 
regression results suggested that the overall model was not significant, y7(8) = 7.25, p > .05. 
In addition, none of the school-level characteristics or variables was associated with school 
response status (p > .05 for all indicators; table A3). 


Table A2. Means and standard deviations of school-level indicators, by survey 
response status, 2016/17 


Responding schools Nonresponding schools 
Standard Standard 
Indicator Mean deviation Mean deviation 
Percentage math proficiency 86 5 24 83 10 29 
Percentage reading proficiency 83 6 24 81 9 29 


Percentage of English learner students, students 
with an Individualized Education Program, and 


students eligible for free or reduced-price lunch? 37 9 23 36 14 26 
Percentage college/career ready 42 t 24 41 12 29 
Graduation rates (percent) 96 4 24 94 12 29 
Attendance rates (percent) 94 2 25 93 4 29 
College enrollment rates (percent) 73 11 25 71 10 28 
School size (N of students) 390.5 335.5 25 352.4 301.2 29 


a. The percentage of English learner students, students with an Individualized Education Program, and stu- 
dents eligible for free or reduced-price lunch were summed to create a single variable because of skewness 
present in the individual variables. 


Note: Pairwise deletion was applied to missing data. 


Source: lowa Department of Education. 
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Table A3. Results of logistic regression examining school-level indicators 
predicting survey participation 


95 percent 

Standard Standard- confidence interval 
Indicator Vitel g ized beta Lower Upper 
Percentage math proficiency 22.31 13.64 0.52 -4.43 49.04 
Percentage reading proficiency -10.03 13.84 -0.22 -37.15 17.09 
Percentage English learner students 41.93 89.55 0.63 -133.58 217.44 
Percentage eligible for free or reduced-price lunch 3.85 5.95 0.12 -7.82 15.52 
Percentage with an Individualized Education Program 17.45 15:53 0.15 -12.98 47.87 
Percentage college/career ready -4.78 10.10 -0.16 -24.58 15.02 
Graduation rate (percent) -4.58 13.41 -0.05 -30.87 21.70 
Attendance rate (percent) 22.14 25.18 0.24 -27.22 71.49 
School size (N of students) 0.002 0.002 0.21 -0.002 0.005 
College enrollment (percent) 4.87 5.05 0.16 -5.02 14.76 
Constant -31.57 26.29 — -83.10 19.96 


Note: N = 55 schools. School-level information was not present in the Common Core of Data 2015/16 dataset for 
two schools that were not in operation at the time of data collection. The model was not statistically significant, 
F(8, 39) = .65. All regression coefficients were not statistically significant (p > .05). The percentage of English 
learner students, students with an Individualized Education Program, and students eligible for free or reduced-price 
lunch were summed to create a single variable because of skewness present in the individual variables. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017 and 
data from the lowa Department of Education. 


Do low response rates in a school reflect a sample biased toward more frequent users 
of technology? This question is based on the premise that teachers who seldom use tech- 
nology are less likely to complete a survey about their technology use because they do not 
perceive the survey as being relevant to their curriculum or instruction. If this were the 
case, it would suggest the possibility of a biased sample in schools with low response rates. 
The possibility for bias is further underscored by the fact that the teacher response rate 
across all schools was 66 percent, with a range of 23 percent to 100 percent. Nine of the 26 
schools in the sample had a teacher response rate of less than 60 percent. 


One way to test for this source of bias is to observe whether schools with low teacher 
response rates tend to have higher average scores for variables related to technology use. 
High nonresponse in such schools would lead to a biased sample of teachers who use tech- 
nology more frequently. To this end, the study team tested the correlation of the school- 
level teacher response rate with the four scales related to technology use, the correlation 
of most relevance to the question of bias involving the frequency of using technology. The 
school-level teacher response rate was not statistically significantly correlated with any of 
the four scales related to technology use or perceptions about technology (table A4). 


Do response rates differ as a function of subject area taught? Finally, one characteristic 
that may affect teachers’ willingness to respond is the subject that they teach. If teachers 
of certain subjects are less likely to perceive the importance of educational technology and 
use it less frequently, they may be less likely to complete the survey. Their nonresponse may 
thus bias the results. For this reason, the study team examined whether the nonresponse 
rate differed as a function of subject taught. The study team compared the subjects taught 
by responders (as indicated by their survey responses) to information on subjects taught by 
all teachers in each school (publicly available through school websites). Of the 26 schools 
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Table A4. Correlations, means, and standard deviations between school-level 
teacher response rate and scales related to technology use 


Standard 
Scale Correlation Mean deviation 
Frequency of computer use# 19 Deft 16 
Perceived benefits of educational technology .08 3.82 .25 
Perceived self-efficacy for instructional use of technology® .09 3.75 .20 
Perceived self-efficacy for technology use® -.05 1.67 18 


Note: Correlation coefficients were not statistically significant (p > .05). The mean school response rate was 
66.53, and the standard deviation was 19.94. 


a. Analysis was performed on a raw score. 
b. Analysis was performed on a summarized scale score. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 


from which at least one teacher responded to the survey, 22 schools provided publicly 
available information about their teachers and the subjects they teach. Subsequent non- 
response analyses were restricted to the 22 schools with publicly accessible data. 


The study team categorized all the subjects into eight groups based on the nature of the 
subjects: four core subjects (math, English language arts, social studies, and science), career 
and technical education subjects (including computer programming and vocational tech- 
nology), arts, physical education, and world languages. In the survey, fewer than one in 
seven teachers (13.5 percent) selected multiple subjects. Half of those teachers (7 percent) 
selected one core subject and one or more other subjects and were assigned to groups by 
the core subject they selected; the other half selected more than one core subject and were 
excluded from the analysis because it was difficult to assign them to a single group. In 
addition, one teacher reported more than two noncore subjects, and that record also was 
excluded from the analysis. One additional teacher’s answer did not fall under the eight 
categories (psychology), so that response was excluded from the analysis as well. 


The study team conducted a chi-square test to compare the response rates for these eight 
subjects. There was a significant association between subject taught and whether a teacher 
responded to the survey [y’(7) = 73.16, p < .05; figure Al]. Post hoc tests indicated that 
teachers in core subjects were more likely to respond to the survey than career and tech- 
nical education teacher and arts teachers. Arts teachers also were less likely to respond to 
the survey than physical education teachers. Therefore, the higher nonresponse from these 
teachers may bias the survey results. For this reason, the study team disaggregated the find- 
ings to highlight variations as a function of subject taught. 


Data analysis 


Because respondents were grouped into different schools, the similarity among members 
of a school may be larger than the similarity among schools, which is called a clustering 
effect of schools. In other words, teachers within the same school may be more similar to 
each other than teachers at different schools are similar to each other. Such a clustering 
effect, if not properly controlled for (for example, by using hierarchical modeling), may 
cause biased estimates of the associations among independent and dependent variables 
and can potentially lead to a Type I error—a false positive conclusion. However, if the 
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Figure A1. Core subject teachers were more likely to respond to the survey than 
non-core subject teachers 
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Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 


differencs among schools are no greater than the differences among teachers within the 
same school, performing hierarchical modeling is unnecessary (Desai, Bryson, & Robin- 
son, 2013). 


To determine the magnitude of the clustering effects of schools, the study team estimat- 
ed intraclass correlation coefficients (ICCs) among independent and dependent variables 
with school as a clustering factor. As a rule of thumb, an ICC of .06 or below indicates 
homogeneity among schools. The study team found that ICCs ranged from .007 to .051 
among different school and teacher characteristics throughout the outcome variables, 
indicating that the schoollevel difference was unlikely to contribute to the teacherlevel 
difference among the outcome variables. Therefore, the subsequent analyses included only 
teacherlevel data that did not account for school-level clustering effects. 


A series of between-group response difference tests was performed to answer the research 
questions in sequence. For the four 21st century skills that did not have reliable Rasch 
scores, the study team calculated the percentage of teachers from different groups (cor 
responding to the variables of subject taught, years of teaching experience, school size, 
and school Title I status) that emphasized each skill. The study team then calculated chi- 
square values for each of these four variables. A significant chi-square would indicate a 
significant difference among the groups within a variable regarding the extent to which 
teachers emphasized a skill. When there was a significant chi-square value for a specific 
variable, the study team performed post hoc tests to examine the extent to which different 
groups of teachers differed from each other.* 


There are different approaches to performing the post hoc tests for nonparametric statistics 
—for example, the pairwise comparison approach and the partitioning approach (Sharpe, 
2015). The pairwise comparison approach separates the groups into several 2 x 2 tables 
and compares the responses from the entities in pairs. However, because this procedure 
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introduces multiple contingency tables, it reduced the sample to small subsets, potentially 
leading to small cell values for some tables. As a rule of thumb, for a cell with a relatively 
small observed or expected value, pairwise approaches often are not reliable. Therefore, 
this approach is not recommended for a large number of groups with small expected cell 
values in the 2 x 2 contingency tables. In addition, the pairwise approach also increases 
the risk of Type I error (inflated alpha) while making multiple comparisons, which should 
be controlled for using a Bonferroni adjusted z test by dividing the overall significance p 
value by the number of contingency tables. Because this study involved 524 teachers in 
eight subjects, the pairwise approach would end up with twenty-eight 2 x 2 contingency 
tables and some expected cell values smaller than 5. Therefore, this is not an appropri- 
ate approach for testing groups of teachers by subject taught. However, teachers’ years of 
teaching experience had only four groups associated with six 2 x 2 contingency tables. 
Therefore, the study team decided that the pairwise approach with Bonferroni correction 
was appropriate for testing groups of teachers by their teaching experience. 


Another approach that involves dividing large contingency tables into a set of smaller 
2 x 2 contingency tables is the partitioning approach, which was deemed appropriate by 
the study team for testing groups of teachers by subject taught. With this approach, the 
full chi-square contingency table was partitioned into several 2 x 2 contingency tables 
corresponding to the degrees of freedom in the full table. Thus, for the chi-square analyses 
involving subject taught, there were seven contingency tables, constructed by ordering 
the groups from highest to lowest by percentage of respondents. The first contingency 
table compared the two highest groups, the second contingency table compared the third 
highest group to the sum of the two highest groups, the third contingency table compared 
the fourth highest group to the sum of the three highest groups, and so on. Bonferroni-ad- 
justed z tests also were applied to the analysis because of the increased risk of a Type I error. 


For the remaining topic areas with reliable sum scores, the study team estimated grouping 
effects for teachers’ subject taught, years of teaching experience, school size, and school 
Title I status on these topic areas using analysis of variance. If the grouping effect was 
significant for a specific grouping variable on an outcome scale, Bonferroni post hoc pair- 
wise comparison tests were performed to examine which two groups scored statistically 
differently on the scale. Detailed results of the mean difference tests and post hoc tests can 
be found in appendix D. 
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Appendix B. Descriptive statistics 


This appendix summarizes descriptive statistics about the composition of the study’s 
sample. 


Table B1. Number of teacher respondents, by teacher and school characteristics 


Percent of 

Characteristic Number respondents Standard error 

All teachers 524 100 na 
“Subject taught 

Arts 23 4.4 0.9 

Career and technical education 40 7.6 alee 

English language arts 77 14.7 1.5 

Math 69 13:2 1.5 

Other@ 142 204 1.9 

Physical education 25 4.8 1.0 

Science 71 13.5 1-5 

Social studies 51 9.7 1.3 

World languages 26 5.0 0.9 
“Years of teaching experience 

3 or fewer 75 14.3 1:5 

4-9 97 18.5 Lf 

10-19 159 30.3 2.0 

20 or more 161 30.7 2.0 

Missing 32 6.1 1.0 
Schoo! size 

Small (300 or fewer students) 214 40.8 2:1. 

Medium (300-500 students) 94 17.9 1 

Large (501 or more students) 216 41.2 2:2 
School Title | status 

No 363 69.3 2.0 

Yes 153 29.2 2.0 

Missing 8 15 0.5 
School locate 

Rural 289 55.2 2.2 

Town 180 34.4 2.1 

City 47 9.0 1.2 

Missing® 8 1:5 0.5 


na is not applicable. 
Note: Percentages may not sum to 100 because of rounding. 


a. Includes special education teachers, psychology teachers, teachers with multiple subject areas, and teach- 
ers with no response, which were not included in the subsequent analysis. 


b. School locale information was not present in the Common Core of Data 2015-16 dataset because schools 
were not in operation at the time of data collection. 


Source: Survey data collected by the Central Rivers Area Education Agency in 2017 (teacher characteristics); 
U.S. Department of Education, National Center for Education Statistics, 2016 (school characteristics). 
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Table B2. Number of participating schools, by size, Title | status, and locale 


Percent of 
participating 

Characteristic schools Standard error 

All schools 26 100 na 
Schoo! Size 

Small (300 or fewer students) 16 61.5 9.7 

Medium (300-500 students) 4 15.4 7.2 

Large (501 or more students) 6 23.1 8.4 
Schoo! Title | status 

No 19 73.1 8.9 

Yes 6 23.1 8.4 

Missing? zl 3.9 3.8 
School locate 

Rural 19 73.1 8.8 

Town 5 19.2 7.9 

City al 3.9 3.8 

Missing? il 3.9 3.8 


na is not applicable. 
Note: Percentages may not sum to 100 because of rounding. 


a. School locale information was not present in the Common Core of Data 2015-16 dataset because schools 
were not in operation at the time of data collection. 


Source: U.S. Department of Education, National Center for Education Statistics, 2016. 


Appendix C. Survey response frequencies 


This appendix summarizes response frequencies for individual survey items and for groups 
of items reported together. 


Table C1. Percentage of teachers reporting frequency of student use of classroom 
technology, by teacher and school characteristics 


Every few Almost 
Characteristic N Never months Monthly Weekly CEN 
All teachers 507 2.8 3.0 4.9 20.9 68.4 
“Subject taught 
Arts 23 30.4 13.0 26.1 13.0 17.4 
Career and technical education 40 0.0 0.0 0.0 15.5 84.5 
English language arts 77 1.3 0.0 11.3 22.1 75.3 
Math 69 1.5 8.7 15.9 24.6 49.3 
Physical education 25 16.0 16.0 8.0 32.0 28.0 
Science 71 0.0 0.0 0.0 15.5 84.5 
Social studies 51 0.0 0.0 2.0 15.7 82.4 
World languages 26 0.0 0.0 0.0 34.6 65.4 
“Years of teaching experience 
3 or fewer 75 4.0 1.3 9.3 22.7 62.7 
4-9 97 2:1 2.1 3.1 22.7 70.1 
10-19 159 2.5 4.4 1.9 20.8 70.4 
20 or more 161 3.1 3.4 6.8 18.6 68.3 
ES 2 ee 
Small (300 or fewer students) 198 3.0 3.0 4.6 14.1 75.3 
Medium (301-500 students) 94 ao. 2:1 6.4 21.3 69.2 
Large (501 or more students) 215 3.3 3.3 4.7 27.0 61.9 
School Title | status 
No 347 3.2 3.8 4.0 20.8 68.3 
Yes 153 2.0 1.3 7.2 21.6 68.0 


Note: Results are based on a single item asking “How often do your students use computer devices (desk- 
tops, laptops, tablets) in class?” Percentages may not sum to 100 because of rounding. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 
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Table C2. Percentage of teachers asking students at least monthly to engage in 
technology use for collaboration, by number of technology uses and teacher and 
school characteristics 


Number of different technology uses for collaboration 
Characteristic N (0) 1 2 3 4 


All teachers 524 30.3 19.1 22.1 24.6 3.2 
“Subject taught 
Arts@ 23 43.5 39.1 17.4 0.0 0.0 
Career and technical education” 40 30.0 15.0 17:5 35.0 2.5 
English language arts® 77 15.6 16.9 24.7 41.6 1:3 
Math@ 69 63.8 20.3 11.6 4.4 0.0 
Physical education? 25 44.0 24.0 16.0 8.0 8.0 
Science” 71 19.7 23.9 28.2 21.1 7.0 
Social studies” 51 11.8 13.7 35.3 37.3 2.0 
World languages® 26 26.9 11.5 34.6 19.2 ere 
“Years of teaching experience 
3 or fewer® ¢ 75 25.3 17.3 24.0 28.0 5.3 
4-94 97 22.7 15:5 32.0 25.8 4.4 
10-19° ¢ 159 25.8 20.8 23.9 25.8 3.8 
20 or more® 161 37.9 21.7 17.4 22.4 0.6 
Schoo! size 
Small (300 or fewer students) 214 34.6 17.3 19.6 24.8 3.7 
Medium (300-500 students) 94 21.3 20.2 24.5 31.9 2.1 
Large (501 or more students) 216 30.1 20.4 25.0 21.3 3.2 
School Title | status 
No 363 31.1 19.6 21.2 25.1 3.0 
Yes 153 30.1 17.7 24.8 24.8 2.6 


Note: Categories refer to the number of technology uses for collaboration, out of four, that teachers asked 
students weekly or monthly to use. Data are raw survey results. Percentages may not sum to 100 because of 
rounding. 


a. Group means with footnote a are not statistically different from each other. 
b. Group means with footnote b are not statistically different from each other. 
c. Group means with footnote c are not statistically different from each other. 
d. Group means with footnote d are not statistically different from each other. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 
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Table C3. Percentage of teachers asking students at least monthly to engage in 
technology use for communication, by number of technology uses and teacher and 
school characteristics 


Number of different technology uses for communication 


Characteristic 0 1 2 3 4 5 
All teachers 524 47.5 26.0 16.8 5.2 1:7 1.3 1.5 
“Subject taught 
Arts@ 23 60.9 17.4 17.4 4.4 0.0 0.0 0.0 
Career and technical education® » 40 42.5 32.5 17.5 2.5 2.5 2:5 0.0 
English language arts® 77 32.5 22.1 23.4 14.3 6.5 0.0 1.3 
Math? 69 75.4 21.7 2.9 0.0 0.0 0.0 0.0 
Physical education® >: ° 25 68.0 16.0 4.0 8.0 0.0 0.0 4.0 
Science > 71 32.4 38.0 19.7 5.6 0.0 1.4 2.8 
Social studies® ° 51. 37.3 31.4 23.5 3.9 2.0 2.0 0.0 
World languages® > 26 34.6 38.5 11.5 7.7 0.0 3.9 3.9 
“Years of teaching experience 
3 or fewer 75 42.7 24.0 24.0 4.0 2.7 1.3 1.3 
4-9 97 41.2 28.9 15.5 9.3 3.1 0.0 2.1 
10-19 159 50.9 25.2 13.8 5.0 1.3 2.5 1.3 
20 or more 161 48.5 29.2 15.5 3.7 1.2 0.6 1.2 
Schoo! size 
Small (300 or fewer students) 214 50.5 22.4 16.8 5.6 1.4 2.3 0.9 
Medium (300-500 students) 94 40.4 31.9 19.2 5.3 1.1 2.1 0.0 
Large (501 or more students) 216 47.7 26.9 15.7 4.6 2.3 0.0 2.8 
School Title status 
No 363 49.9 25.6 15.7 4.4 1.9 4.4) 4.7 
Yes 153 43.1 26.8 19.0 7.2 1.3 2.0 0.7 


Note: Categories refer to the number of technology uses for communication, out of six, that teachers asked 
students weekly or monthly to use. Data are raw survey results. Percentages may not sum to 100 because of 
rounding. 


a. Group means with footnote a are not statistically different from each other. 
b. Group means with footnote b are not statistically different from each other. 
c. Group means with footnote c are not statistically different from each other. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 
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Table C4. Percentage of teachers asking students at least monthly to engage in 
technology use for creativity, by number of technology uses and teacher and school 
characteristics 


Number of different technology uses for creativity 


Characteristic 


All teachers 524 53.8 23.9 11.3 8.0 3.1 
“Subject taught 
Arts@ 23 52.2 26.1 17.4 4.4 0.0 
Career and technical education? 40 35.0 21.9) 20.0 10.0 7.5 
English language arts® 77 42.9 45.5 7.8 1.1 2.6 
Math? 69 85.5 11.6 4.5 0.0 1.5 
Physical education? 25 68.0 20.0 0.0 4.0 8.0 
Science? 7 32.4 25.4 15.5 22.5 4.2 
Social studies? 51 43.1 29.4 21.6 3.9 2.0 
World languages® 26 53.9 23.1 11.5 11.5 0.0 
“Years of teaching experience 
3 or fewer® ¢ 75 46.7 28.0 16.0 8.0 1.3 
4-9° 97 46.4 18.6 15.5 12.4 7.2 
10-19° ¢ 159 52.2 25.8 12.0 7.6 2.5 
20 or more? 161 60.3 26.1 6.2 5.6 1.9 
Schoo! size 
Small (300 or fewer students) 214 51.9 22.4 14.0 8.9 2.8 
Medium (300-500 students) 94 45.7 27.7 11.7 13.8 alga 
Large (501 or more students) 216 59.3 23.6 8.3 4.6 4.2 
School Title status 
No 363 54.0 25.1 11.0 6.9 3.0 
Yes 153 54.9 21.6 10.5 a 24. 2.0 


Note: Categories refer to the number of technology uses for creativity, out of four, that teachers asked stu- 
dents to use. Data are raw Survey results. Percentages may not sum to 100 because of rounding. 


a. Group means with footnote a are not statistically different from each other. 
b. Group means with footnote b are not statistically different from each other. 
c. Group means with footnote c are not statistically different from each other. 
d. Group means with footnote d are not statistically different from each other. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 
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Table C5. Percentage of teachers asking students at least monthly to engage in 
technology use for critical thinking, by number of technology uses and teacher and 
school characteristics 


Number of different technology uses for critical thinking 


Characteristic 1 2 
All teachers 524 30.2 26.2 15.5 13.0 15.3 
“Subject taught 
Arts@ 23 69.6 13.0 8.7 8.7 0.0 
Career and technical education” 40 25.0 17.5 12.5 22.5 22.5 
English language arts? 77 19.5 40.3 23.4 15.6 1.3 
Math@ 69 59.4 13.0 14.5 7.3 5.8 
Physical education? 25 60.0 24.0 4.0 4.0 8.0 
Science” 7 8.5 19.7 18.3 12.7 40.9 
Social studies” 51 15.7 37.3 13.7 23.5 9.8 
World languages® 26 42.3 38.5 11.5 3.9 3.9 
“Years of teaching experience 
3 or fewer 75 28.0 29.3 16.0 14.7 12.0 
4-9 97 17.5 33.0 14.4 11.3 23.7 
10-19 159 28.9 22.6 13.8 15.7 18.9 
20 or more 161 37.3 25.5 18.0 11.8 7.5 
Schoo! size 
Small (300 or fewer students) 214 29.0 24.3 15.9 13.6 17:3 
Medium (300-500 students) 94 28.7 24.5 12.8 16.0 18.1 
Large (501 or more students) 216 31.9 28.7 16.2 iE 12.0 
School Title | status 
No 363 27.6 29.5 15.4 13.8 13.8 
Yes 153 37.9 19.0 15.7 Bs Be 16.3 


Note: Categories refer to the number of technology uses related to critical thinking, out of four, that teach- 
ers asked students weekly or monthly to use. Data are raw survey results. Percentages may not sum to 100 
because of rounding. 


a. Group means with footnote a are not statistically different from each other. 
b. Group means with footnote b are not statistically different from each other. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 
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Table C6. Percentage of teachers reporting agreement or disagreement that 
technology enhances student learning, by teacher and school characteristics 


Strongly Strongly 
Characteristic N disagree Disagree Neutral Agree agree 
All teachers 524 0.4 1.0 21.2 51.0 26.5 
“Subject taught 
Arts2> 23 0.0 4.4 39.1 39.1 17.4 
Career and technical education?” 40 0.0 0.0 15.0 57.5 27.5 
English language arts*” 77 1.3 0.0 23.4 50.7 24.7 
Math? 69 15 4.4 27.5 46.4 20.3 
Physical education®” 25 0.0 0.0 24.0 60.0 16.0 
Science” pal 0.0 0.0 14.1 53.5 32.4 
Social studies?” yl 0.0 2.0 19.6 47.1 31.4 
World languages*” 26 0.0 0.0 15.4 53.9 30.8 
“Years of teaching experience 
3 or fewer 75 0.0 1.3 20.0 52.0 26.7 
4-9 97 0.0 2.1 13.4 57.7 26.8 
10-19 159 0.0 0.0 22.6 49.1 28.3 
20 or more 161 4:2 1.2 23.0 50.9 23.6 
Schoo! size 
Small (300 or fewer students)° 214 0.5 0.5 16.4 51.4 31.3 
Medium (301-500 students)* 4 94 1.1 1.1 16.0 57.5 24.5 
Large (501 or more students)? 216 0.0 1.4 28.2 47.7 22.1 
School Title | status 
No 363 0.3 1.1 22.0 48.8 27.8 
Yes 153 0.7 0.7 19.0 56.9 22.9 


Note: Data are scale score summaries derived from responses to three survey items. The study team per- 
formed post hoc tests on subgroup average scores to examine whether reported perceived benefits differed by 
subject taught, years of teaching experience, school size, or school Title | status. Percentages may not sum to 
100 because of rounding. 


a. Group means with footnote a are not statistically different from each other. 
b. Group means with footnote b are not statistically different from each other. 
c. Group means with footnote c are not statistically different from each other. 
d. Group means with footnote d are not statistically different from each other. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 
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Table C7. Percentage of teachers reporting agreement or disagreement that 
they are able to integrate technology with instruction, by teacher and school 
characteristics 


Strongly Strongly 
Characteristic N disagree Disagree Neutral Agree agree 
All teachers 524 0.6 6.3 153) 48.1 29.8 
Subject taught 
Arts 23 0.0 8.7 17.4 60.9 13.0 
Career and technical education 40 0.0 ED 20.0 42.5 30.0 
English language arts 77 1:3 5.2 18.2 49.4 26.0 
Math 69 1.5 5.8 24.6 39.1 29.0 
Physical education 25 0.0 8.0 4.0 68.0 20.0 
Science 71 0.0 5.6 5.6 40.9 47.9 
Social studies 51 0.0 5.9 7.8 51.0 35.3 
World languages 26 0.0 7.7 41:5 53.9 26.9 
“Years of teaching experience 
3 or fewer 75 1.3 1.3 13.3 53.3 30.7 
4-9 97 0.0 1.0 12.4 43.3 43.3 
10-19 159 0.0 3.8 14.5 50.9 30.8 
20 or more? 161 4:2 13:7 17.4 46.0 21.7 
Schoo! size 
Small (300 or fewer students) 214 0.9 6.1 13.1 49.1 30.8 
Medium (300-500 students) 94 0.0 6.4 13.8 48.9 30.9 
Large (501 or more students) 216 0.5 6:5 18.1 46.8 28.2 
School Title | status 
No 363 0.6 7.7 15.7 46.0 30.0 
Yes 153 0.7 3.3 14.4 53.6 28.1 


Note: Data are scale score summaries derived from responses to four survey items. The study team per- 
formed post hoc tests on subgroup average scores to examine whether reported perceived self-efficacy for 
instructional use of technology differed by subject taught, years of teaching experience, school size, or school 
Title | status. Percentages may not sum to 100 because of rounding. 


a. The means among the groups differed significantly with no statistically significant pairwise difference in 
groups. 


b. The scale score is statistically different from that of groups that do not include footnote b. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 
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Table C8. Percentage of teachers reporting access to computers, by frequency of 
access and teacher and school characteristics 


Less More 
than half than half 


of the of the 
Characteristic Never Rarely time time Always 


All teachers 519 0.4 1.2 1.2 4.1 93.3 
Arts 23 4.4 13.0 0.0 4.4 78.3 
Career and technical education 39 0.0 0.0 2:5 0.0 97.5 
English language arts (3 0.0 1:3 0.0 2.6 96.1 
Math 64 0.0 0.0 2.9 4.4 92.8 
Physical education 25 4.4 4.4 8.7 8.7 73.9 
Science 69 0.0 0.0 0.0 1.4 98.6 
Social studies 51 0.0 0.0 0.0 0.0 100.0 
World languages 23 0.0 0.0 0.0 11.5 88.5 
“Years of teaching experience 
3 or fewer 70 0.0 13 0.0 5.3 93.3 
4-9 90 1.0 1.0 0.0 52 92.8 
10-19 151 0.0 0.0 1.9 2:5 95.6 
20 or more 141 0.6 2.6 1.9 5.1 89.8 
Schoo! size 
Small (300 or fewer students) 200 0.5 0.5 0.5 4.7 93.9 
Medium (300-500 students) 90 4.1 0.0 2.4. 44 95.7 
Large (501 or more students) 194 0.0 2.4 1.4 4.7 91.5 
School Title status 
No 335 0.3 1.4 1.7 3.9 92.8 
Yes 142 0.7 0.7 0.0 4.0 94.7 


Note: Results are based on a single item asking “How frequently can you access computer devices (desktop, 
laptop, tablet) for your students’ use?” Percentages may not sum to 100 because of rounding. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 
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Table C9. Percentage of teachers reporting that school Internet filters prevent 
access to needed websites, by frequency of blocking and teacher and school 
characteristics 


Less More 
than half than half 
of the of the 
Characteristic Never Rarely time time Always 
All teachers 507 13.8 49.3 27.4 6.7 2.8 
“Subject taught 
Arts 23 13.0 30.4 43.5 8.7 4.4 
Career and technical education 40 7.0 65.4 19:2 3.9 3.9 
English language arts 77 13.7 56.9 21.6 5.9 2.0 
Math 69 10.4 49.4 32.5 5.2 2.6 
Physical education 25 24.0 44.0 24.0 4.0 4.0 
Science 71 18.8 37.5 33.3 6.3 4.2 
Social studies 51 11.3 50.7 26.8 7.0 4.2 
World languages 26 10.4 49.4 32.5 5.2 2.6 
“Years of teaching experience 
3 or fewer 75 25.3 41.3 26.7 5:3: 1.3 
4-9 97 12.4 54.6 24.7 5.2 3.1 
10-19 159 10.1 52.8 27.0 6.9 3.4 
20 or more 161 12.4 45.3 31.1 8.1 3.1 
School size 
Small (300 or fewer students) 198 9.1 53.5 24.8 yp 5.6 
Medium (300-500 students) 94 18.1 41.5 28.7 10.6 1.4 
Large (501 or more students) 215 16.3 48.8 29.3 4.7 0.9 
School Title status 
No 347 13.0 52.5 24.2 6.6 3.8 
Yes 153 16.3 42.5 33.3 7.2 0.7 


Note: Results are based on a single item asking “When using the school’s Internet, how often do school filters 
prevent you from accessing websites you need for classes?” Percentages may not sum to 100 because of 
rounding. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 
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Table C10. Percentage of teachers reporting on perceived technical support 
quality, by perceived quality of support and teacher and school characteristics 


Below Above 
Characteristic N Poor average Average average Excellent 
All teachers 524 0.6 0.6 34.9 47.3 16.6 
“Subject taught 
Arts 23 0.0 0.0 47.8 39.1 13.0 
Career and technical education 40 0.0 0.0 25.0 52.5 22.5 
English language arts 77 0.0 0.0 33.8 45.5 20.8 
Math 69 0.0 0.0 39.1 55.1 5.8 
Physical education 25 0.0 4.0 16.0 52.0 20.0 
Science 71 2.8 0.0 33.8 46.5 16.9 
Social studies bi. 0.0 0.0 35.3 47.1 17.7 
World languages 26 0.0 0.0 50.0 34.6 15.4 
“Years of teaching experience 
3 or fewer 75 0.0 0.0 46.7 46.7 6.7 
4-9 97 0.0 1.0 40.2 40.2 18.6 
10-19 159 0.0 1.3 31.5 48.4 18.9 
20 or more 161 1.9 0.0 30.4 48.5 19.3 
Schoo! size 
Small (300 or fewer students)? 214 0.0 0.0 29.0 51.9 19.2 
Medium (301-500 students)*” 94 1.1 1.1 35.1 43.6 19.2 
Large (501 or more students) 216 0.9 0.9 40.7 44.4 13.0 
Schoo! Title | status 
No 363 0.3 0.0 33.3 49.9 16.5 
Yes 153 1.3 1:3 37.3 43.1 17.0 


Note: Data are scale score summaries derived from responses to seven survey items. The study team per- 
formed post hoc tests on subgroup average scores to examine whether reported perceived technical support 
quality differed by subject taught, years of teaching experience, school size, or school Title | status. Percent- 
ages may not sum to 100 because of rounding. 


a. Group means with footnote a are not statistically different from each other. 
b. Group means with footnote b are not statistically different from each other. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 
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Table C11. Percentage of teachers reporting on perceived quality of technology- 
focused professional development, by perceived quality and teacher and school 
characteristics 


Below Above 
Characteristic N Poor average Average average Excellent 
All teachers 524 2.5 5.9 55.3 28.6 7.6 
“Subject taught 
Arts 23 4.4 13.0 39.1 30.4 13.0 
Career and technical education 40 7.5 5.0 52.5 30.0 5.0 
English language arts 77 1.3 5.2 55.8 29.9 7.8 
Math 69 1.5 11.6 46.4 37.7 2.9 
Physical education 25 4.0 4.0 48.0 24.0 20.0 
Science 71 1.4 4.2 59.2 28.2 7.0 
Social studies 51 0.0 0.0 60.8 31.4 7.8 
World languages 26 7.7 7.7 46.2 34.6 3.9 
“Years of teaching experience 
3 or fewer 75 0.0 1.3 54.7 30.7 13.3 
4-9» 97 2A. 5.2 65.0 21.7 6.2 
10-1925 159 3.1 3.8 54.1 30.2 8.8 
20 or more> 161 2.5 9.9 51.6 31.7 4.4 
Schoo! Size 
Small (300 or fewer students) 214 2.8 3.3 56.1 29.4 8.4 
Medium (300-500 students) 94 2.1 10.6 53.2 30.9 3.2 
Large (501 or more students) 216 2.3 6.5 55.6 26.9 8.8 
School Title | status 
No 363 2.5 5.8 55.4 29.2 t2 
Yes 153 2.6 6.5 55.6 26.8 8.5 


Note: Data are scale score summaries derived from responses to three survey items. The study team 
performed post hoc tests on subgroup average scores to examine whether the reported perceived quality of 
professional development differed by subject taught, years of teaching experience, school size, or school Title 
| status. Percentages may not sum to 100 because of rounding. 


a. Group means with footnote a are not statistically different from each other. 
b. Group means with footnote b are not statistically different from each other. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 
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Table C12. Percentage of teachers reporting technology-focused professional 
development time, by hours spent and teacher and school characteristics 


33 or 
1-8 9-16 17-32 more 
Characteristic hours hours hours hours 
All teachers 524 10.1 55.2 10.9 17.9 5.9 
Arts 23 4.4 60.9 13.0 21.7 0.0 
Career and technical education 40 17.5 42.5 15:0 15.0 10.0 
English language arts 77 9.1 50.7 18.2 18.2 3.9 
Math 69 13.0 56.5 10.1 14.5 5.8 
Physical education 25 4.0 44.0 20.0 20.0 12.0 
Science rae 7.0 59.2 7.0 23.9 2.8 
Social studies 51 7.8 66.7 3.9 15.7 5.9 
World languages 26 11.5 57.7 11.5 11.5 7.7 
“Years of teaching experience 
3 or fewer 75 9.3 50.7 10.7 25.3 4.0 
4-9 97 12.4 56.7 13.4 14.4 3.1 
10-19 159 8.8 56.6 8.8 17.6 8.2 
20 or more 161 9.3 57.8 10.6 14.9 7.5 
Bie SS Se Se SS eee 
Small (300 or fewer students) 214 15.4 50.5 10.3 19.2 47 
Medium (300-500 students) 94 5.3 52.1 16.0 20.2 6.4 
Large (501 or more students) 216 6.9 61.1 9.3 15.7 6.9 
School Title | status 
No 363 9.9 57.0 10.2 17.4 5.5 
Yes 153 14.4 51.6 13:4. 18.3 5.9 


Note: Data are scale score summaries derived from responses to three survey items. The study team did 
not perform post hoc tests on subgroup average scores because the main effects of between-group differ- 
ence tests were not significant across teacher and school characteristics. Percentages may not sum to 100 
because of rounding. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 
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Table C13. Percentage of teachers reporting on frequency of technology 
discussions, by frequency of discussion and teacher and school characteristics 


Less More 
than half than half 


of the of the 
Characteristic Never Rarely time time Always 


All teachers 524 2.7 16.0 31.9 35.9 13.6 
Arts@ 23 8.7 34.8 30.4 26.1 0.0 
Career and technical education” 40 2.5 10.0 27.5 45.0 15.0 
English language arts® 77 1.3 15.6 33.8 33.8 15.6 
Math?» 69 1.5 23.2 31.9 30.4 13.0 
Physical education?» 25 4.0 16.0 40.0 32:0 8.0 
Science” 71 0.0 12.7 25.4 45.1 16.9 
Social studies’ 51 2.0 7.8 31.4 37.3 21.6 
World languages*” 26 3.9 15.4 38.5 30.8 11.5 
“Years of teaching experience 
3 or fewer 75 2.7 12.0 30.7 44.0 10.7 
4-9 97 1.0 14.4 32.0 35.1 17.5 
10-19 159 3.4 12.0 32.1 34.0 18.9 
20 or more 161 3.7 20.5 31.1. 35.4 9.3 
Schoo! size 
Small (300 or fewer students) 214 2:3 15.4 29.4 38.3 14.5 
Medium (300-500 students) 94 2.4. 18.1 28.7 38.3 12.8 
Large (501 or more students) 216 3:2 T5:f 35.7 32.4 13.0 
“School Title | status 
No 363 1.9 15.7 33.6 36.4 12.4 
Yes 153 4.6 17.7 27.5 34.6 15.7 


Note: Data are scale score summaries derived from responses to three survey items. The study team per- 
formed post hoc tests on subgroup average scores to examine whether the reported prevalence of technology 
discussions differed by subject taught, years of teaching experience, school size, or school Title | status. 
Percentages may not sum to 100 because of rounding. 


a. Group means with footnote a are not statistically different from each other. 
b. Group means with footnote b are not statistically different from each other. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 
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Appendix D. Summary of tests for mean differences 


This appendix summarizes results of analyses testing differences across groups of teachers 
by subject taught, years of teaching experience, school size, and school Title I status. 


Table D1. Chi-square tests of goodness of fit for proportions of teachers using 
technology to support 21st century skills, by skill and teacher and school 
characteristic 


Degrees of freedom 


Between Within 
Skill and characteristic groups groups Chi square p value 


Subject taught 


7 374 36.9 <.0014 
Years of teaching experience 3 488 4.3 ns 
School size 2 521 0.2 ns 
School Title | status 4 514 1.7 ns 
Communication 
Subject taught 7 374 69.7 <.001 
Years of teaching experience 3 488 13.4 <.005 
School size 2 521 3.0 ns 
School Title | status al 514 0.4 ns 
Creativity 
Subject taught 7 374 44.4 <.001 
Years of teaching experience 3 488 16.4 <.001 
School size 2 521 5.8 ns 
School Title | status 1 514 0.4 ns 
| Criteal thining 
Subject taught 7 374 54.6 <.001 
Years of teaching experience 3 488 5.5 ns 
School size 2 521 2.8 ns 
School Title | status ul 514 0.0 ns 


ns is not significant. 


Note: Chi-square goodness of fit tests examine the proportion of teachers within different categories who ask 
students to engage in two or more technology practices at least monthly for each 21st century skill, compared 
with the overall proportion. p values less than .05 indicate statistically significant between-group differences. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 


Table D2. Mean difference tests, by survey topic and teacher and school 
characteristic 


Degrees of freedom 


Within 
Topic and characteristic Between groups groups F value p value 


Subject taught 


it 374 2.4 <.05 
Years of teaching experience 3 488 1.2 ns 
School size 2 521 6.0 <.01 
School Title | status 1 514 0.2 ns 


Subject taught it 374 22 <.05 
Years of teaching experience 3 488 10.3 <.001 
School size 2 B24: 1.0 ns 
School Title | status 1. 514 0.5 ns 


Subject taught 7 374 0.5 ns 
Years of teaching experience 3 488 0.8 ns 
School size 2 521 1.4 ns 
School Title | status 1 514 0.2 ns 


Subject taught iA 358 0.8 ns 
Years of teaching experience 3 468 4.3 <.01 
School size 2 500 1.3 ns 
School Title | status 1 493 0.03 ns 


Subject taught 7 374 3.9 <.001 
Years of teaching experience 3 488 2.6 ns 
School size 2 521 0.5 ns 
School Title | status al 514 0.3 ns 


Subject taught t 373 1.8 ns 
Years of teaching experience 3 487 2.0 ns 
School size 2 520 5.0 <.01 
School Title | status A. 513 1.9 ns 


ns is not significant. 


Note: The F statistic is based on Fisher’s test of between-group difference. Pairwise deletion was applied to 
missing data. p values less than .05 indicate statistically significant between-group differences. 


Source: Authors’ analysis of survey data collected by the Central Rivers Area Education Agency in 2017. 
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Appendix E. Survey items analyzed in the study 


This section presents the items in the survey that were included in the analysis for this 


study. 


1. How often do your students use computer devices (desktops, laptops, tablets) in class? 


(Response options: almost daily, weekly, monthly, every few months, and never) 


2. How frequently do you ask a majority of your students to do the following? (Response 


options: weekly, monthly, every few months, and never). 


Collaborate using online documents (for example, Dropbox, Google Docs). 
Collaborate online with classmates. 

Collaborate online with students at other schools. 

Collaborate online with you. 

Post schoolwork online (for example, use ePortfolios). 

Use web tools to receive online information (for example, Twitter, news feeds). 
Write for an online audience (for example, reviews, blog posts, comments). 
Receive feedback digitally from classmates. 

Receive feedback digitally from someone other than you (for example, an outside. 
expert, a student in another class or school). 

Chat online (for example, Skype, Google Hangout, FaceTime). 

Use a digital camera (photo or video). 

Develop multimedia presentations using technology. 

Create art, music, movies, or webcasts using technology. 

Create online models, simulations, or animations. 

Conduct research online. 

Take measurements or do experiments using technology. 

Identify and solve authentic problems using technology. 

Collect and analyze data using technology. 


3. Indicate how strongly you agree or disagree with the following statements. (Response 


options: strongly agree, agree, are neutral, disagree, and strongly disagree) 


Technology use in the classroom enhances student learning. 

I think that learning is more engaging when using technology. 

My school encourages technology use for teaching and learning. 

I want to learn more about effective technology use for teaching and learning. 


4. Indicate how strongly you agree or disagree with the following statements. (Response 


options: strongly agree, agree, are neutral, disagree, and strongly disagree) 


I learn technology easily. 

When I am confronted with a technology-related problem, I usually find good 
solutions. 

I easily find new technologies to meet my teaching goals. 

I feel confident managing a classroom where students are using technology. 
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10. 


How many hours have you spent in the past 12 months participating in the following 

types of educational technology professional development? (Response options: 33 or 

more hours, 17 to 32 hours, 9 to 16 hours, 1 to 8 hours, none) 

e Formal PD sponsored by the school or district (for example, in-service days, 
mentoring). 

e Formal PD organized by someone other than the school or district (for example, 
degree programs, conferences). 

e Informal PD organized by someone other than the school or district (for example, 
blogs, social media). 


What was the quality of the following types of educational technology professional 

development you’ve completed in the past 12 months? (Response options: excellent, 

above average, average, below average, poor, I didn’t do this) 

e Formal PD sponsored by the school or district (for example, in-service days, 
mentoring). 

e Formal PD organized by someone other than the school or district (for example, 
degree programs, conferences). 

e Informal PD organized by someone other than the school or district (for example, 
blogs, social media). 


How often are the following statements true for you? (Response options: all of the 
time, more than half of the time, less than half of the time, rarely, never) 

e My department or grade-level team discusses technology use at meetings. 

e I discuss technology use during my evaluations. 

e I discuss technology use during class observations or visits. 


Rate the quality of support for the following. (Response options: excellent, above 

average, average, below average, poor, not applicable) 

e Internet speed. 

e¢ Computer devices (desktops, laptops, or tablets). 

e Interactive whiteboards or display devices (for example, LCD projectors, large 
monitors). 

e Support for problems disrupting instruction. 

e Answers to routine questions. 

e Instructional technology planning. 

e Hardware repair. 


When using the school’s Internet, how often do school filters prevent you from access- 
ing websites you need for classes? (Response options: never, rarely, less than half of the 
time, more than half of the time, all of the time) 


How frequently can you access computer devices (desktop, laptop, tablet) for your stu- 


dents’ use? (Response options: all of the time, more than half of the time, less than 
half of the time, rarely, never) 
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. What subject(s) do you teach? Check all that apply. 
e Computer science, programming, or technology. 
e Language arts or English. 

e = Math. 

e Physical education. 

e Performing arts (dance, music, theater, ...). 

e Psychology. 

e Science. 

e Social studies or history. 

e World languages. 

e Visual arts. 

e Vocational technology. 


. How long have you been teaching? 
e 3 or fewer years. 

e 4to9 years. 

e 10 to 19 years. 

e 20 or more years. 


Notes 


The study team gratefully acknowledges the input from members of a stakeholder advisory 
group: Michelle Cowell (from the Central Rivers Area Education Agency, Iowa) and Chad 
Kliefoth and Janice Mertes (both from the Wisconsin Department of Public Instruction). 


1. The Central Rivers AEA is one of nine Iowa area education agencies that provide 
consulting, leadership, professional development, and resource-sharing services to 
public and nonpublic districts in Jowa. Central Rivers AEA serves 53 public districts 
in 18 mostly rural counties in central and northeast central Iowa. 

2. The Central Rivers AEA, previously known as AEA 267, is one of nine Iowa area 
education agencies that provide professional development, leadership, consulting, and 
resource-sharing services to public and nonpublic districts in Iowa. Central Rivers 
AEA serves 53 public districts in 18 mostly rural counties in central and northeast 
central Iowa. These 53 districts account for 14.4 percent of districts in the state. Of the 
57 high schools in those districts, 40 are in rural locales, accounting for 6 percent of 
rural high schools in the state. 

3. Schools receiving Title I funds either as a schoolwide program or as a targeted assis- 
tance program were compared with schools not receiving any Title I funds. Title I 
provides financial assistance through state educational agencies to local educational 
agencies and public schools with high numbers or percentages of students from low- 
income households to help ensure that all students meet challenging state academic 
content and student academic achievement standards. 

4. These frequencies are based on scale scores and represent each respondent’s most 
likely response across all three items about time spent in professional development (see 
box 2). 

5. A chi-square test was not necessary for Title I status because there were only two 
groups in this variable. 
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